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How To Find Us - Virtual Reality and Immersive Visualization 



IT Center 

Kopernikusstrafte 6 (Erweiterungsbau) 
Email: weyers@vr.rwth-aachen.de 
{nowke, schnorr}@vr.rwth-aachen.de 

www.vci.rwth-aachen.de 





How To Find Us - Computer Graphics and Multimedia 



Informatik E3 
Ahornstr. 

Email: kobbelt@informatik.rwth-aachen.de 
issak.lim@cs.rwth-aachen.de 

www.vci.rwth-aachen.de 
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Who are your Teachers in this Course? 



•Computer Graphics & Multimedia 

■ Prof. Dr. Leif Kobbelt (Lecture) 

Scientific Visualization, Scalar and Vector Field 
Visualization, Data Types, Grid Interpolation 

■ Isaak Lim, M.Sc. 

— Exercises 




•Virtual Reality & Immersive Visualization 

■ Dr. Benjamin Weyers (Lecture) 

— Information Visualization, Rendering, Virtual 
Reality, Perception 

■ Dipl. -Inform. Christian Nowke 

■ Andrea Schnorr, M.Sc. 

— Exercises 
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Who Should Attend this Course? 



• Computational Engineering Science (B.Sc.) 

• 5. Semester, Pflichtfacher 

• Simulation Sciences (M.Sc.) 

• Mandatory Courses, Semester 1 
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The Simulation Loop 



Problem 




Simulator 



Simulation 

Simulation Results | 



Prob lem solved 




Application 
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Closer look to (Visual) Analysis in the Simulation Loop 




RAW DATA 




Perception 



IMAGE DATA 




Visualization 




VIS DATA 



Rendering 
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Why Data Analysis & Visualization? 




JUQUEEN IBM Blue Gene/Q System, JSC, 
Forschungszentrum Julich GmbH 
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raw data 

Gigabyte-Terabyte-Petabyte 



• Generating insight from data requires data analysis. 

Hamming: „The purpose of Computing is insight, not numbers!" 
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Why Data Analysis & Visualization? 




juqueen ibm Blue Gene/Q System, jsc, Gigabyte-Terabyte- Petabyte Visualization 



Forschungszentrum Julich GmbH 

• Generating insight from data requires data analysis. 

Hamming: „The purpose of Computing is insight, not numbers!" 

• Visualization is (primarily) being used for the analysis process 
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EU Flagship: The Human Brain Project 



1 Billion Euros Funding for the next 10 years 
■ A major goal: 

■ Simulation of large biologically realistic neural networks 

■ ... up to the human brain scale (10 11 neurons, 10 15 synapses) 
Extremely dynamic data at high temporal resolution 
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Why Data Analysis & Visualization? 





11011110110101 1 
_ 0 01 0 1 0 01 0 1 001 1 ^ l! 
010010101001010 
101001010101010 
010101110101011 



010111100 . . . 



juqueen IBM Blue Gene/Q System, jsc, Gigabyte-Terabyte-Petabyte 

Forschungszentrum Julich GmbH 



raw data 



• Generating insight from data requires data analysis. 

Hamming: „The purpose of Computing is insight, not numbers!" 



• Visualization is (primarily) being used for the analysis process 

• Amount of raw data is rapidly increasing: Finer grids, 3-D, time-variant 

• Explorative versus confirmative analysis, Virtual Reality 



H Dr.-Ing. Benjamin Weyers | Virtual Reality & Immersive Visualization | WS 2015/16 | 
Course on Data Analysis and Visualization 




Visual Computing 
Institute 




UNVERSnY 




The aixCAVE @ RWTH 
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The “Virtual Windtunnel” 







VR Applications in CES @ RWTH 




Motors & Turbines 
Twin Extruders 
Nasal Airflow 
Bood Flow 
Crash Simulations 
Material Science 
Pig housing 



FEV GmbH, VR Group 





> 2015/16 | 



The Visualization Pipeline 






Filtering 



Mapping 



Rendering 




• Filtering 

* Mapping 



• Rendering 



Preprocessing of data 

e.g. subsampling, dimension reduction... 

Transformation to “primitives” (graphical objects) 
e.g. polygonal meshes 

Transformation of render-primitives 
to output images 
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Course Outline 



20.10.2015 

(VI) Introduction & Basics - Rendering (Weyers) 

Course Introduction, Organization, Visualization and Rendering Pipeline 

27.10.2015 

(V2) Basics - Rendering (Nowke) 

Shading, Scan Conversion, Texture Mapping 

03.11.2015 

(U1) (Schnorr/Nowke) 

(V3) Basics - Perception (Weyers) 

Sensory and Abitrary Perception, Human Eye, Color Models, Gestalt Psychology 

10.11.2015 

(U2) (Schnorr/Nowke) 

(V4) Basics - Visualization (Kobbelt) 

Data types, grid interpolation and integration, & Scalar field visualization: transfer function 
design, iso-contouring, volume rendering 
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Course Outline (cont.) 



17.11.2015 
(U3) (Lim) 

(V5) Scientific Visualization (Kobbelt) 

Vector field visualization: glyphs, flow lines, streak lines 

24.11.2015 
(U4) (Lim) 

(V6) Large Data Analysis (Kobbelt) 

01.12.2015 
(U5) (Lim) 

(V7) Large Data Analysis (Kobbelt) 

08.12.2015 
(U6) (Lim) 

(V8) Immersive Visualization (Weyers) 

Explorative analysis of simulation datasets in Virtual Reality 
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Course Outline (cont.) 



15.12.2015 

(U7) (Schnorr/Nowke) 

(V9) Information Visualization (Weyers) 

Introduction to information visualization and Data Types 

22.12.2015 — NO COURSES --- 

12.01.2016 

(U8) (Schnorr/Nowke) 

(VI 0) Information Visualization (Weyers) 

Visualization and representation of values and relations 

19.01.2016 

(U9) (Schnorr/Nowke) 

(VII) Information Visualization (Weyers) 

Visualization of graphs and node link diagrams 



B JM Dr.-lng. Benjamin Weyers | Virtual Reality & Immersive Visualization | WS 2015/16 | 

Course on Data Analysis and Visualization 




Visual Computing 
Institute 




UNVERSnY 



Course Outline (cont.) 



26 . 01.2016 

(U10) (Schnorr/Nowke) 

(VI 2) Information Visualization (Weyers) 

Graph drawing and layout algorithms 

02 . 02.2016 

(U11) (Schnorr/Nowke) 

(VI 3) Information Visualization (Weyers) 

Graph drawing and layout algorithms, limitations of information visualization 

09 . 02.2016 

(U12) (Schnorr/Nowke) 

Questions and Answers (Lim/Schnorr/Nowke/Weyers) 

Exams - Check in Campus! 

First Exam: Thursday, 18.02.2016, 13:15 to 15:30 (AH V) 

Second Exam: Tuesday, 22.03.2016, 10:00 to 12:00 (5056) 
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Organizational Stuff 



• Literature: „Handapparat“ in Computer Science Library 

• Script? Course contents, e.g., in A. Watt, CG, und K.-F- Kraiss, AMMI 

• Exercises: Just BEFORE the lessons 

• Material: L2P (Exercise and Lecture) 

• Miscellaneous, announcements, exercises: L2P 

• Relevant for Exam: 

• Content presented in lecture 

• Content presented and discussed in exercises! 
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Literature for the Rendering / Virtual Reality Part 



Books (*Handapparat Informatik-Bibliothek): 

• *D. Bowman et al. 3D User Interfaces. Addison-Wesley 

• *K. M. Stanney. Handbook of Virtual Environments. Erlbaum 

• *M. Slater et al. Computer Graphics & Virtual Environments. Addison-Wesley 

• *G. Burdea, P. Coiffet. Virtual Reality Technology. John Wiley & Sons 

• *K.-F. Kraiss (Ed.). Advanced Man Machine Interfaces. Springer 

• R.S. Kalawski. The Science of Virtual Reality and Virtual Environments. Addison 
Wesley 

• F. Dai. Lebendige virtuelle Welten. Springer Verlag 

• J.D. Foley, A. van Dam, S.K. Feiner, J.F. Hughes. Computer Graphics - Principles 
and Practice. Addison Wesley 

• K.D. Tonnies, H.U. Lemke. 3D-Computergrafische Darstellungen. Oldenburg Verlag 

• *A. Watt. 3D Computer Graphics. Addison Wesley 

• Conferences: 

• IEEE VR, IEEE Vis 

• IPT, EGVE 

• ACM Conferences (SIGGRAPH, VRST, VRCAI, ...) 
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Course on Data Analysis and Visualization 

Basics in Computer Graphics 
The Rendering Pipeline 

-- Benjamin Weyers & Christian Nowke -- 
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Topics - Basics in Computer Graphics / Rendering Pipeline 



* Rendering Pipeline 

* Representation of rigid objects 

* Transformations 

* Culling 

* Projection 

* Clipping 

* Hidden Surface Removal 

* Shading 

* Scan Conversion 

* Texture Mapping 

* Graphics Hardware 

* Ray Tracing (optional topic) 
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View Volume 




eye position clipping plane 

(viewpoint) 
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Overview - Rendering Pipeline 





Transformation: 

Position und orientation 
of objects 




Transformation: 



WKS, SKS 

* 



Clipping 




Projection 



Culling 



l 

Shading 



Hidden Surface 
Removal 
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Steps - Rendering Pipeline 



1 . Transformations: 

- Describe position & orientation of objects within a scene 

- Transform local coordinates of objects into a single world coordinate 
system (WKS) 

- Optional: Transform into a view Coordinate system (SKS): 
viewpoint = origin 

2. Culling: 

- Remove invisible polygons 

3. (Perspective) Projection: 

- Transform Vertices into screen coordinate system 
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Steps - Rendering Pipeline 



4. Clipping: 

— Truncate geometry outside the view volume 

5. Scan Conversion: 

- Transform the polygonal model into a set of pixels (picture elements) 

6. Hidden Surface Removal (HSR): 

— Remove invisible pixels, covered by other objects 

7. Shading: 

— Find color (intensity) values of single pixels based on an illumination 
model 



1 
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Remarks - Rendering Pipeline 



• Step order can vary 

• 5-7 are normally combined into a single step (Pixel Loop) 

• Pipeline principle: Speed up 

• 2 is a special case of 6: Performance! 
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Culling versus Hidden Surface Removal 




E |V Dr.-lng. Benjamin Weyers | Virtual Reality & Immersive Visualization | WS 2015/16 | 

Course on Data Analysis and Visualization 




RWIHAACHBI 

UNVERSIIY 



Topics - Basics in Computer Graphics / Rendering Pipeline 



* Rendering Pipeline 

* Representation of rigid objects 

* Transformations 

* Culling 

* Projection 

* Clipping 

* Hidden Surface Removal 

* Shading 

* Scan Conversion 

* Texture Mapping 

* Graphics Hardware 

* Ray Tracing (optional topic) 
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Polygonal Representation - Object Representation 



Approximation of object geometry by a grid of planar, polygonal facets 




smooth curves and surfaces require many points! 






watch for valid polygons! (no problems with triangles) 





invalid 



Pictures: Foley et al., Watt 
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Data Structure for Polygonal Representation - Object Representation 



• List of 3-D coordinates (x, y, z) (polygon vertices) 

• Edges implicitly 

• Additional information: normals of polygons or polygon 
vertices for shading, color, material, ... 

• Hierarchical structure of single geometries (scene 
graph) 

• Polygons are rendered independently from each other 

• Drawback: Redundant rendering of common edges 

• Alternative: explicit storage of edges - each edge is a 
list of 4 parameters: 2 adjacent vertices and 2 
polygons 





Picture: Schuhmann 

H Dr.-Ing. Benjamin Weyers | Virtual Reality & Immersive Visualization | WS 2015/16 | 

Course on Data Analysis and Visualization 




Visual Computing 
Institute 



wmmai 

ivERsmr 



Polygonal Representation - Benefits 



• Hardware Rendering 

• Efficient Shading algorithms 

• Arbitrary geometries 

• Trade Off: Accuracy (number of polygons) 

versus 

speed (frame rate, polygons per second) 
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Alternatives - Object Representation 





• Functional representation, e.g., sphere: x 2 + y 2 + z 2 = r 

• Constructive solid Geometry 

• Bezier Patches 



• Octrees 

• Volume Rendering 

(See lessons on visualization) 
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Topics - Basics in Computer Graphics / Rendering Pipeline 



* Rendering Pipeline 

* Representation of rigid objects 

* Transformations 

* Culling 

* Projection 

* Clipping 

* Hidden Surface Removal 

* Shading 

* Scan Conversion 

* Texture Mapping 

* Graphics Hardware 

* Ray Tracing (optional topic) 
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Overview - Rendering Pipeline 





Transformation: 

Position und orientation 
of objects 




Transformation: 



WKS, SKS 

* 



Clipping 




Projection 



Culling 



l 

Shading 



Hidden Surface 
Removal 
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Basic Operations - Transformation 
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Basic Operations (II) - Transformation 



Polygonal model: Just transform the vertices! 
Conventions: 

• Points are described as column or row vectors P ( x , y, z) 

• Transformations are described as matrices: 

- Translation P' = P + D (D = translation vector) 

- Rotation P' = P * R (R = rotation matrix) 

- Scaling P' = P* S (S= scale matrix) 

- Shearing P' = P * Sh (Sh = shear matrix) 
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Homogeneous Coordinates - Transformation 



Inconsistency: 

Translation is vector-vector-addition, all other operations are 
matrix vector-multiplication 

Solution: Homogeneous coordinates - Increase dimension 
by 1 

P(x,y,z)~ P(X,Y,Z,w), with x =^y=^ = ^ 

Convention: w =1 











UNVERSnY 
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Translation - Transformation 



P=PT 




o' 

0 
0 

1 



x —x+T x , y —y+Ty, z—z + T ^ 
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Scaling - Transformation 



P=PS 








=x-S x , y'=y-Sy, z=z-S z 
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Rotation - Translation 
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l 



cos# sin# 0 0 

-sin# cos# 0 0 

0 0 10 

0 0 0 1 , 

Rotation about the z-axis: 

x'—xcosO— ysin# , y'=ys\n6+ycos6 , z'=z 
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Application - Transformation 



Concatenation of transformations by matrix-matrix-multiplication: 
Representation of arbitrary linear transformations by a single matrix 

• Operations on points of a single coordinate system 

• Convert one coordinate system into another: 

- Local Object Coordinate Systems 

- Coordinate Systems for groups of objects 

- World Coordinate System 

- View Coordinate System 

- Screen Coordinate System 
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Topics - Basics in Computer Graphics / Rendering Pipeline 



* Rendering Pipeline 

* Representation of rigid objects 

* Transformations 

* Culling 

* Projection 

* Clipping 

* Hidden Surface Removal 

* Shading 

* Scan Conversion 

* Texture Mapping 

* Graphics Hardware 

* Ray Tracing (optional topic) 
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Overview - Rendering Pipeline 





Transformation: 

Position und orientation 
of objects 




Transformation: 



WKS, SKS 

* 



Clipping 




Projection 



Culling 



l 

Shading 



Hidden Surface 
Removal 
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Algorithm - Culling 



• Remove invisible polygons to speed up rendering (Culling, backface 
elimination) 

• Compare polygon orientation with “line of sight” vector 

Simple, fast „algorithm“ 

Polygon is invisible if and only if the angle between the polygon normal 




Backface Elimination of a Cube - Culling 



sichtbar 



Auge re- 
position 



un sichtbar 




Picture based on Watt 
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Computation - Culling 



r 



0\ N p ,N 






<90 



j 



N p • N >0 



cos 6 \= 



Determination of N and N P \ 

• N is the position vector in the viewing coordinate system 

• Calculate N P from 3 (non-collinear) polygon vertices 




N 

N 
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Culling - Remarks 



• Compute N P s offline before the simulation starts, or store them with the 
model 

• Reuse the normal vectors for shading 

• At an average, half of the polygons of a polyhedron are invisible: 
Culling eliminates them at the very beginning of the rendering process 
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Topics - Basics in Computer Graphics / Rendering Pipeline 



* Rendering Pipeline 

* Representation of rigid objects 

* Transformations 

* Culling 

* Projection 

* Clipping 

* Hidden Surface Removal 

* Shading 

* Scan Conversion 

* Texture Mapping 

* Graphics Hardware 

* Ray Tracing (optional topic) 
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Overview - Rendering Pipeline 





Transformation: 

Position und orientation 
of objects 




Transformation: 



WKS, SKS 
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Clipping 




Projection 
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Shading 
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Removal 
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Definition - Projection 



• General: 

A projection is a function from n-dimensional space to (n-l)-dimensional 
space 

• Computer Graphics: 

- 3-D ^^2-D (screen, projection plane, viewing window) 

- Projection plane is flat: straight lines are mapped to straight lines 
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Perspective versus Parallel Projection - Projection 



Perspective projection 

• Projectors (projection beams) 
meet in the center of projection 
(eye position, viewpoint) 

• Objects get „smaller“ with rising 
distance from the center of 
projection 

• view volume is a pyramid 



• Parallel projection 

• Projectors (projection beams) 
are parallel, center of projection 
is in infinitum 

• Does not correspond to natural 
viewing experience 

• view volume is a cube 
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View Volume - Projection 
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Perspective Shortening - Projection 



Picture: Watt 
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A very Simple Projection - Assumptions 



• Perspective projection 

• No ..diagonal" projection: Straight line from the center of projection to 
the center of the screen is parallel to the normal vector of the viewing 
window 

• Constant distance dfrom the center of projection to the screen 

• Computation: 

- View coordinate system 

Normal vector of the viewing window is parallel to the z-axis 
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A very Simple Projection - Intersection Point 



For each point P(x v , y v , z v ) of the object geometry in 3-D space, we are 
looking for the intersection point P'(x s , y s ) of the projector (straight line 
from viewpoint to P) with the view plane. 
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A very Simple Projection - Computation 



Computation via similar triangles 
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A very Simple Projection - Projection Matrix 



Homogeneous coordinates: 



X Xy , Y y y ^ X Zy , w ^ 

(X Y Z w) = [xv y v z v ^y Pro j mit 

0 O' 
0 0 

‘ i 

0 oj 



Proj 



"1 0 
0 1 
0 0 
0 0 
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Topics - Basics in Computer Graphics / Rendering Pipeline 



* Rendering Pipeline 

* Representation of rigid objects 

* Transformations 

* Culling 

* Projection 

* Clipping - will be skipped in this course 

* Hidden Surface Removal 

* Shading 

* Scan Conversion 

* Texture Mapping 

* Graphics Hardware 

* Ray Tracing (optional topic) 
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View Volume - Clipping 
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Topics - Basics in Computer Graphics / Rendering Pipeline 



* Rendering Pipeline 

* Representation of rigid objects 

* Transformations 

* Culling 

* Projection 

* Clipping 

* Hidden Surface Removal 

* Shading 

* Scan Conversion 

* Texture Mapping 

* Graphics Hardware 

* Ray Tracing (optional topic) 
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Z-Buffer - Hidden Surface Removal 



Z-Buffer 

• Special memory on graphics hardware, 1 entry for every pixel 

• Updated with the highest z value of 3-D object points that cover its pixel 

• Accuracy depends on 

- Length of memory words („depth“) (usually 24 bits/pixel) 

- Z-value of front and back clipping plane 

• Compare points lying on the same projector (OpenGL: parallel projection!) 
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Topics - Basics in Computer Graphics / Rendering Pipeline 



* Rendering Pipeline 

* Representation of rigid objects 

* Transformations 

* Culling 

* Projection 

* Clipping 

* Hidden Surface Removal 

* Shading 

* Scan Conversion 

* Texture Mapping 

* Graphics Hardware 

* Ray Tracing (optional topic) 
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Shading - Reflection on Surfaces 




a) Direct reflection b) Multi-surface reflection c) Subsurface reflection 

i 1 1 1 1 





1) Mirror reflection 2) Directed diffuse 
(perfect specular reflection) reflection 




3) Diffuse Reflection 
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Shading - Phong Reflection Model 



Linear combination from 3 components: 

• Diffuse 

• Specular 

• Ambient 



Reflection coefficients: kr*k *k 

a ’ s ’ a 




Diffuse component: ^ = / ^ CQS q = / ^ ( L . N ) /, = k d £ h, n {h • #) 

Specular component: = /.^ COS " Q = I t k s (R-V )” 

U Index for surface roughness 

Perfect mirror: zz — > co( Ray Tracing: Recursion) 



Ambient component: / = I a k a 



Overall intensity: / = + /. (k d (L ■ N)+ k^R-V)") 
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Shading - Diffuse vs. Specular 







0 


# 0 


0 








0 


s A 


• 


0 



1 
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Shading - Phong Reflection Model - Errors 



Picture: Watt 



Light 

source 




Specular 

highlight 



Erroneous specular 
highlight in 
shadow area 



Area in 
shadow 
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Overview - Rendering Pipeline 





Transformation: 

Position und orientation 
of objects 




Transformation: 



WKS, SKS 

* 



Clipping 




Projection 



Culling 



l 

Shading 



Hidden Surface 
Removal 
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Shading - What is Shading? 



Shading 

Definition : (Incremental, interpolative) 

Application of a reflection model on polygons by calculation of 
intensities at polygon vertices and interpolation of these values for 
the inner points 

CG: Phong reflection model 

1 . Flat Shading 

2. Gouraud Shading 

3. Phong Shading 
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Shading - Flat Shading vs. Gouraud Shading 



* Flat Shading 

No interpolation, all inner points of a polygon get the same intensity 

* Gouraud Shading 

(Bilinear) interpolation of the inner points from the vertex intensities 





Pictures: Foley et al. 

H Dr.-Ing. Benjamin Weyers | Virtual Reality & Immersive Visualization | WS 2015/16 | 

Course on Data Analysis and Visualization 




Visual Computing 
Institute 



wmmai 

IVERSIIY 





Gouraud Shading - Bilinear Interpolation 



done 

already 
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Gouraud Shading - Diffuse Component 



• Calculate the vertex normal vector as average from the adjacent 
polygon normal vectors (offline!) 

• Calculate the vertex intensities according to Phong model 

• Interpolation process (integrated in scan conversion) in scan line 
order 

• Interpolate intensities at the intersection points of scan line and polygon 
edges from the vertex intensities 

K = (A (y, -y*)+h (>’, - >’> )) 

yi-y 2 

h = (A (>’, - y 4 ) - + h (y, - y, )) 

yi-y* 



Interpolate intensities for the inner points along the scan line from the 
intensities at the intersection points 
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Gouraud Shading - Some Problems 



• No highlights in the middle of polygons (example: virtual table) 




• Corrugations are smoothed out 
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Phong Shading 



Interpolation of vertex normal vectors instead of intensities 

Get an individual normal vector for each pixel as an approximation 
of the “real” normal vector on a curved surface 



N 



l 




vertex 



Gouraud 



Phong 



highlight 



interior 



Gouraud 



Phong 
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Phong Shading - Interpolation of Normal Vectors 



N a = (Nfa-yJ+Nfa-y,)) 
y i~y 2 

N b =-^—(N l (y s -y 4 )+N 4 (y l -y s )) 

yi~y 4 



n s = ( N a (x b -x s )+N b (x s - x a )) 

X b~ X a 

• Phong: three times more complex than Gouraud 

• Also: Calculate intensity at each pixel 

• Speed up: Combine Gouraud and Phong Shading (Interpolate normal 
vector for every second pixel and interpolate intensity for the other 
pixels) 

• Phong shading never realized in graphics hardware (?) - can be 
simulated with textures 
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Gouraud Shading vs. Phong Shading 



Pictures: Foley et al. 




Simple Gouraud Shading 



Phong Shading 
with Specular Highlights 



fl 
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Topics - Basics in Computer Graphics / Rendering Pipeline 



* Rendering Pipeline 

* Representation of rigid objects 

* Transformations 

* Culling 

* Projection 

* Clipping 

* Hidden Surface Removal 

* Shading 

* Scan Conversion will be skipped, but Supersampling! 

* Texture Mapping 

* Graphics Hardware 

* Ray Tracing (optional topic) 



i 
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Display Resolution 




XGA 1 999 
VGA 1 997 
NTSC 1995 

CGA 1993 

Picture: Burdea et al. 
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Today: 

• SXGA, 1 280x1 024 

• UXGA, 1600x1200 

• WUXGA, 1920x1200 



See also: 

• Color depth 

• Anti-Aliasing 
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Scan Conversion - Anti-Aliasing 



Pictures: IT Center RWTH 




1 sample 4 samples 16 samples 



fl 
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Multisampling on XVR 4000 



Courtesy of C. Gonzalez, Sun Microsystems 



1) 1-16 Samples, random distribution 
that changes with each pixel, 
calculated in 64x64 subpixel area 



2) Samples are collected and filtered 
within an area shaped as circle and 
5x5 pixels large 








UNIVERSfTY 
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Multisampling on XVR 4000 (cont.) 



Courtesy of C. Gonzalez, Sun Microsystems 



3) All samples within the circle 
are weighted, normalized 
and accumulated 



4) The filter curve is programmable. 
User can adapt filter characteristics 
to the application 







CD 



o 

CD 

0 



1 
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Overview - Rendering Pipeline 





Transformation: 

Position und orientation 
of objects 




Transformation: 



WKS, SKS 

* 



Clipping 




Projection 



Culling 



l 

Shading 



Hidden Surface 
Removal 







m 
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Topics - Basics in Computer Graphics / Rendering Pipeline 



* Rendering Pipeline 

* Representation of rigid objects 

* Transformations 

* Culling 

* Projection 

* Clipping 

* Hidden Surface Removal 

* Shading 

* Scan Conversion 

* Texture Mapping 

* Graphics Hardware 

* Ray Tracing (optional topic) 
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Texture Mapping - Motivation 



Texture Mapping: Usage of textures in CG (Catmull 1974) 

Increase realism of virtual scenes: 

• Render the structure of surfaces without increasing geometry 
complexity 

• Integrate real photos into the virtual scene 

• Simulate mirror reflections without ray tracing 

• Render semi-transparent surfaces (e.g., glass) 

Scientific Visualization 

• LIC (linear integral convolution) 

• Imaging in medicine: Render volume by textures 
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Texture Mapping - Modulation of Color 



Picture: Foley et al. 




H 
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Texture Mapping - Parameter Modulation 



U 




Texture Map 



Color 

- Functional or procedural 

- Randomized 

- Photo 

Specular and diffuse component 
(Environment Mapping) 

Transparency (CC- Blending) 

Perturbation of the normal vector 
(Bump Mapping) 
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Texture Mapping - Alpha Blending 






■ 




Virtual building at Schinkel Straf3e, RWTH Aachen University 

(Generated by IT Center in 1998) 
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Texture Mapping - Environment Mapping 



| 



Picture: Watt 




Texture Mapping - Linear Integral Convolution 




Simulation of air flow within a kitchen 
(here: faster method without convolution - “integrate-and-draw”) 



Picture: IT Center RWTH 
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Texture Mapping - Bump Mapping 



Pictures: Watt 
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Texture Mapping - S-Mapping 



T (u,v) T'(x„ y„ z,): „S-Mapping“ 

Interim objects 

• enclose the virtual object 

• simple mapping functions 

• Arbitrarily oriented plane 

• Cylinder (without top and bottom sides) 

• Sphere 

• Cube (No distortion, but clipping necessary) 



B Dr.-Ing. Benjamin Weyers | Virtual Reality & Immersive Visualization | WS 2015/16 | 
Course on Data Analysis and Visualization 




Visual Computing 
Institute 




UNVERSnY 



From Texture Map to Pixels 



1 . Surface parameterization 

2. Projection (interpolation, see Shading) 




Use 2-phase parameterization to avoid distortions for polygon models: 

a) S-Mapping 

b) O-Mapping 
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S-Mapping on a Cube 
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S-Mapping Example: Cylinder 



S Cylinder • (©> h)^{u,v) = (- (® - © () ), ^ (/? ~ fl () ) 

\c a ) 



, with 



0 o ,/t o : Texture position 

c, d : scaling factors 
r : Cylinder radius 

©:[0,2n] 

h : [O, cylinder height ] 
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Texture Mapping - 0-Mapping 



T \ x i > y t 0(x w ,y w ,zj 

4 strategies: 

1 . Intersection point of the surface normal vector at every position 

(x w ,y w ,z w ) and T(x i9 y i9 Zi) 

2. Intersection point of the interim object at every position 
with the object surface 

3. Projection of the object midpoint onto the interim surface 

4. Mirror reflection of a sight vector at the interim object. Texture seems to walk 
over the object surface when the viewer moves (Environment Mapping). 






B 



Normalenvektor 

der 

Objektpunkte 



Normalenvektor 

der 

Hilfsobjektpunkte 



Objekt- 

zentrum 
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Texture Mapping - Cinema 4D 
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S-Mapping - Effects of Different Interim objects 



Pictures: Watt 







Cube Mapping 
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The Principle of Environment Mapping 



Picture: Tonnies 



Projektion der Umgebung 

auf eine geschlossene Texturflache 




Objekt 




Sichtstrahl 





Projektion der 
Textur auf die 
Oberflache 
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Environment Mapping vs. Ray Tracing 



Picture: Tonnies 



reale 

Szene 



The point you see 
in ray tracing 



The point you see in 



environment mapping 







Objektzentrum 

Jm 



Sichtstrahl 



Betrachter 
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Texture Mapping - Aliasing Example 
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Texture Mapping - Aliasing Effects 



2 extreme situations: 

1 . Viewer is very close to the textured object: Only one texel for many 
pixels (tiling effect) 

2. Textured object is very far from the viewer: Many texels for only one 
pixel (see example on last slide) 



ad 1/2. Texture hierarchies and Mip Mapping 

ad 2. Shrink Wrap Method and Inverse Mapping 
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Texture Hierarchies 














Increasing D 
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Mip Mapping - Trilinear Interpolation 
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Shrink Wrap Method & Inverse Mapping 



• Start from the 4 pixel corners (Inverse 
Mapping) 

• Transform all 4 corners into texture 
coordinates 

• Integrate over the surface in the texture 
map that is covered by the pixel 
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Topics - Basics in Computer Graphics / Rendering Pipeline 



* Rendering Pipeline 

* Representation of rigid objects 

* Transformations 

* Culling 

* Projection 

* Clipping 

* Hidden Surface Removal 

* Shading 

* Scan Conversion 

* Texture Mapping 

* Graphics Hardware 

* Ray Tracing (optional topic) 
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How to Measure Graphics Performance 



• Speed: 

- Number of polygons per second (Attention: Size and shape of polygons, 
shading method, ...) 

- Fill Rate 

- Data transport: Graphics hardware - CPU - main memory 

- Benchmarks: ViewPerf, ... (see WWW) 

• Quality: 

- Memory - Frame Buffer (size, color depth), Z-Buffer (depth), Texture 
Memory, ... 

- Texture Mapping Features 

- Hardware Anti-Aliasing 

- Special Features: Stereo, Vertex & Pixel Shaders, ... 

• Flexibility: 

- Configuration Interface 

- Number of resolutions and screens supported, ... 
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History of Graphics Performance 



Picture:Burdea et al. 
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How to make graphics chips faster... 



• Chip development, processor clock speed 

• Fast and larger busses (PC:AGP, PCI Express) 

• Add more memory: Resolution, color depth, z-buffer depth, amount of 
textures, anti-aliasing 

• Pipelining 

- ..macroscopic" (Rendering Pipe) 

- onChip (float-multiplication) 

• „Real“ parallel (SIMD,...) 

- ..macroscopic" (more Chips) 

- onChip (more ALUs, ...) 

2 strategies: 

• Geometry on CPU, scan conversion on graphics hardware 

• Both on graphics hardware 
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Overview - Rendering Pipeline 





Transformation: 

Position und orientation 
of objects 




Transformation: 



WKS, SKS 

* 



Clipping 




Projection 



Culling 



l 

Shading 



Hidden Surface 
Removal 
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Composition vs. Partition 



Picture: Kraiss, Technische Informatik, RWTH Aachen 




Composition 
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Topics - Basics in Computer Graphics / Rendering Pipeline 



* Rendering Pipeline 

* Representation of rigid objects 

* Transformations 

* Culling 

* Projection 

* Clipping 

* Hidden Surface Removal 

* Shading 

* Scan Conversion 

* Texture Mapping 

* Graphics Hardware 

* Ray Tracing (optional topic) 
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Ray Tracing - Topics 



• What is Ray Tracing? 

• The Basic Algorithm 

• Intersections 

• Light and Reflection 

• Reflection- and Transmission Rays 

• Optimization 

• Quality 

• Speed 
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Overview - Rendering Pipeline 





Transformation: 

Position und orientation 
of objects 




Transformation: 



WKS, SKS 

* 



Clipping 




Projection 



Culling 



l 

Shading 



Hidden Surface 
Removal 
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Ray Tracing -The Idea 



Whitted 1980: 

Create photorealistic images by following single light beams 

Simulate the process of light distribution by the laws of ideal mirror reflection and refraction 



Root: 

Ray optics in physics 
(e.g., Descartes) 

• Occlusion (HSR) 

• Shadows 

• Reflection 

• Refraction 
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Gouraud Shading vs. Ray Tracing 
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The starting situation 



• 3-D scene 

• Objects 

• (Point) light sources 

• Eye position 

• Screen (viewing plane), Pixel 
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First Step: Ray Casting 




• View rays through all pixels of the 
viewing plane 

• Intersection point of the view ray with 
an object 

• Appel (1968) 

Hidden Surface Removal 
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Shadow Feelers I 




• ^Shadow Feelers": Rays from the intersection point to the light sources 

• Calculate intensity at the intersection point (e.g., Phong Reflection 
Model) 
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Shadow Feelers II 




ipl 

iilfi 




• Object between intersection point and light source: Intersection point is 
in the shadow 
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Reflection 





• Intersection point on mirroring object (specular reflection): Calculate 
and follow the reflected ray 
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Reflection/Deflection 
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• Intersection point on a transparent object: Calculate and follow the 
refracted ray 
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Ray Types 



Reciprocity of reflection: Backwards 
Ray Tracing 



Global illumination method 




shadow feeler s 
normal vector 




'l 'reflection vector 

transmission vector 
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Recursion 
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Effect on Recursion depth 
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• • 



The Basic Algorithm - Pseudo Code for View Rays 



Calclmage 

{ 

For (y=0; y<YRES; y++) 

{ 

For (x=0; x<XRES; x++) 

{ 



ViewRay.Start = Viewpoint 
ViewRay.Dir = CalcViewDir(x,y, Viewpoint); 
Colour = TraceRay(ViewRay,0); 

Plot(x,y, Colour); 



} 



} 



} 



m 
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The Basic Algorithm - Pseudo Code for Ray Tracer 



Colours TraceRay (ray: Ray, depth: int) 

{ 

if (depth>MAXDEPTH) return black 
else 



{ 



} 



} 



Object, IntersectionPoint = Schneide Strahl mit alien Objekten und 9 

ermittle nachstgelegenen Schnittpunkt; ■ 
if (Nointersection) return background_color 
else 



LocalColour = Anteile der sichtbaren Lichtquellen;*? 

ReflectedRay = CalcReflectedRay(ray, IntersectionPoint, Object); 9 
RefractedRay = CalcRefractedRay(ray, IntersectionPoint, Object); ■ 
ReflectedColour = TraceRay (ReflectedRay, depth+1); 
RefractedColour = TraceRay (RefractedRay, depth+1); 
return combine(LocalColour, ReflectedColour, RefractedColour); 
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Ray Tracing - Topics 



• What is Ray Tracing? 

• The Basic Algorithm 

• Intersections 

• Light and Reflection 

• Reflection- and Transmission Rays 

• Optimization 

• Quality 

• Speed 
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Intersections - Object Geometries 



• Spheres 

• Planes 

• Polygons 

• Polyhedrons 

• Cubes 

• Quadrics (cylinders, cones, ellipsoids, ...) 

• Compound objects 

(e.g., Constructive Solid Geometry) 

• Strategy: Approximate complex geometries 
by bounding volumes 

„Good“ volumes: spheres, cubes 
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Intersections - Sphere 



^ * D 0 



(l) x s =x 0 +t-x D ,y s =y 0 +t-y D ,z s = z 0 +t-z 



"ftl 




(2) (x s - x M f + (y s - y M f + (z s -z M f = r 



0 



(l) in (2) : 



(x 0 + 1 ■ x D - x M ) 2 + (y 0 + 1 ■ y D - y M ) 2 +(z 0 +t-z D - z M f - r 



A't + B * t + (2 — 0 with 
A = x 2 d + y 2 D + z 2 D = 1 (D normalized) 

B = 2{x d (x 0 - x M )+ y d (y 0 - y M )+ z d (z 0 - z M )) 

c = (*o ~ x m f +U> - y M f + to - Zm f ~ r2 
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Intersections - Sphere (cont.) 




-B±Jb T ^4C 

2 



• Select smaller t 

• if B -4C<0 , the ray misses the sphere 



Intersection point: = (x s , y s , Z s ) = (x 0 + 1 ■ X D , y 0 + 1 ■ y D , Z 0 + 1 ■ Z D ) 



Normal vector: 



f 



N s = 



x s~ x m y s ~y M ys~y M 



\ 



V 



J 
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Ray Tracing - Topcis 



• What is Ray Tracing? 

• The Basic Algorithm 

• Intersections 

• Light and Reflection 

• Reflection- and Transmission Rays 

• Optimization 

• Quality 

• Speed 
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Phong Reflection Model 



Linear combination from 3 components: 

• Diffuse 

• Specular 

• Ambient 



Reflection coefficients: kr*k *k 

a ’ s ’ a 




Diffuse component: ^ = / ^ CQS q = / ^ ( L . N ) /, = k d £ ' # ) 

Specular component: = /.^ COS " Q = I t k s (R-V )” 

U Index for surface roughness 

Perfect mirror: 72 — > oo(Ray Tracing: Recursion) 



Ambient component: / = I a k a 



Overall intensity: / = + /. (L • #)+ k s (R • V )" ) 
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Colours TraceRay (ray: Ray, depth: int) 

{ 



if (depth>MAXDEPTH) return black 



else 



Object, IntersectionPoint = Schneide Strahl mit alien Objekten und 



LocalColour =_Anteile der sichtbaren_Lichtquellen; 

ReflectedRay = CalcReflectedRay(ray, IntersectionPoint, Object);^ 
RefractedRay = CalcRefractedRay(ray, IntersectionPoint, Object);" 
ReflectedColour = TraceRay (ReflectedRay, depth+1); 
RefractedColo ur = T raceRav (RefractedRay. deoth+ 1): 
return combine(LocalColour, ReflectedColour, RefractedColour); 



ermittle nachstgelegenen Schnittpunkt; 



if (Nointersection) return background_color 
else 
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Calculation of Reflection & Transmission Rays 



• Sight (view) vector S 

• reflection vector ^ 

• Transmission vector j 

• Normal vector ^ 

• R\0 S =0 R bzw. cos 6 S = cos 0 R (- S N = N R) 

• j . s * n _ ^1 Snell's Law 

sin 0 T rf 2 

• Algebraic solution R/T = aS + j3N 

• Geometrical solution 




r f (S' + N) 




R = S’ + 2N 
T = r f (S' + N)-N 



mit r f = 






1 



\sf -|S' + A| 
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Algebraic Solution for Transmission Rays 



Equation 1: 
sin 0 c 






Equation 2: 



1 = 



sin 0 T 


02 




sin 6 s ri 


= sin 0 T 




sin 2 0 s 7j 2 


= sin 2 0 T 


— 


(l-cos 2 6 S ) 


rj 2 = (l-cos 2 0 T ) 


— 


(l - cos 2 6 s ) 


rj 2 -1 = cos 2 0 T 





[- N ■ T f 

[- N ■ (ccS + fJN )] 2 

[a cos 6 S - 



TT 

(aS + J3N)- (aS + J3N ) 

a 2 {l ■ l)+2aj3(l ■ N)+ J3 2 (N ■ N) 

a 2 - 2 a/3 cos 6 S + /? 2 



Find solution for c & nd j^id insert into T = aS + f3N 
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The Basic Algorithm - Pseudo Code for Ray Tracer 



Colours TraceRay (ray: Ray, depth: int) 

{ 

if (depth>MAXDEPTH) return black 
else 



{ 



} 



} 



Object, IntersectionPoint = Schneide Strahl mit alien Objekten und 

ermittle nachstgelegenen Schnittpunkt; 
if (Nointersection) return background_color 
else 

{ 

LocalColour = Anteile der sichtbaren Lichtquellen; 

ReflectedRay = CalcReflectedRay(ray, IntersectionPoint, Object); 
RefractedRay = CalcRefractedRay(ray, IntersectionPoint, Object); 
ReflectedColour = TraceRay (ReflectedRay, depth+1); 
RefractedColour = TraceRay (RefractedRay, depth+1); 
return combine(LocalColour, ReflectedColour, RefractedColour); 

} 
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Ray Tracing - Topics 



• What is Ray Tracing? 

• The Basic Algorithm 

• Intersections 

• Light and Reflection 

• Reflection- and Transmission Rays 

• Optimization 

• Quality 

• Speed 
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Optimization - Quality 



• Anti-Aliasing 




• Supersampling: 

• Treat pixels as a plane 

• Send more rays through each pixel 




• Averaging the intensities of single rays leads to pixel color 
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Optimization - Quality (cont.) 






• Anti-Aliasing 

• Adaptive Supersampling: 

• First look at the rays at the 4 pixel corners 

• Does the intensity difference of adjacent rays exceed threshold? 

• Recursive grid refinement (Quad-Trees) 

• Stochastic Ray Tracing: 

• Irregular but uniformly dense distribution of rays 
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Optimization - Speed 



Ray Traycing is slow! 



• Standard resolution SXGA (1280x1024): 1.3 Mio pixels 

• Small scene with 50 objects: 65 Mio intersection points must be 
determined in the first step 

• Algorithm complexity increases exponentially with recursion depth (in 
case of „specular“ objects) 

• Shadow feelers cause additional intersection points (number of light 
sources?) 

• Situation is even harder for Distributed Ray Tracing and Anti-Aliasing 

• Make use of parallelism! 

• Optimize the algorithm for the calculation of single intersection points 

• Reduce the number of intersection points 
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Optimization - Speed (II) 



Optimize the intersection point algorithm 
Bounding Volumes 

• Tradeoff: Accuracy of approximation versus complexity of intersection point 
calculation 



• Option: Stepwise approximation 
Space subdivision 

• Octrees: Nearly the same complexity in the leaves 



T) 




f\ 










U 







• Make use of the coherency for adjacent rays 

• View rays 

• Rays ending at the same plane (see polygons!) 
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Ray Tracing - Summary 



+ Elegant, recursive algorithm for global illumination 

+ Reflection, transmission, occlusion culling, hidden surface removal and shadows 
+ Very realistic images if there are many specular and transparent objects in the scene 



- Local modeling of diffuse reflection 
^ Distributed Ray Tracing 
^ Radiosity 

^ Plausibility versus authenticity: see Radiance! 

Ray Tracing is slow 

> Special hardware for Ray Tracing (e.g., Advanced Rendering Technologies, Cambridge) 

> Anyway: Ray Tracing may be even faster than traditional shading for scenes with 
millions of polygons and a lot of occlusion! 
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Real Time Ray Tracing 




A. Dietrich, I. Wald, P. Slusallek (scene data from 0. Deussen): OpenRT 



28.000 sun flowers, 1 Billion polygons, 6 frames/sec (640x480 pixels), PC-Cluster (24 nodes) 
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Literature 



Where to find the images used in the slides: 



• Foley, van Dam, Feiner, Hughes: Computer Graphics: Principles and Practice. Addison 
Wesley, 1 992 

• Glassner: An Introduction to Ray Tracing, Morgan Kaufmann Publishers, 2000 

• Tonnies, Lemke: 3D-Computergrafische Darstellungen, Oldenbourg Verlag, 1994 

• Watt: 3D Computer Graphics, Addison Wesley, 1992 

• Whitted: An improved illumination model for shaded display, Comm, of the ACM 23(6), 
1980 

• http://www.gris.informatik.tu-darmstadt.de/lehre/vorl_ueb/gdvll/slides/rr-bw.pdf 

• http://www2.inf.fh-bonn-rhein-sieg.de/~ahinke2m/Vorlesung/CGVI03/ 
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